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Abstract 

The literature on programming languages contains an abundance of in- 
formal claims on the relative expressive power of programming languages, 
but there is no framework for formalizing such statements nor for deriving 
interesting consequences. As a first step in this direction, we develop a 
formal notion of expressiveness and investigate its properties. To validate 
the theory, we analyze some widely held beliefs about the expressive power 
of several extensions of functional languages. Based on these results, we 
believe that our system correctly captures many of the informal ideas on 
expressiveness, and that it constitutes a foundation for further research 
in this direction. 

1 Comparing Programming Languages 

The literature on programming languages contains an abundance of informal claims 
on the expressive power of programming languages. Arguments in these contexts 
typically assert the expressibility or non-expressibility of programming constructs 
relative to a language. Unfortunately, programming language theory does not provide 
a formal framework for specifying and verifying such statements. Comparing the set of 
computable functions that a language can represent is useless because the languages in 
question are usually universal; other measures do not exist. The lack of a comparison 
relation makes it impossible to draw any firm conclusions from expressiveness claims 
or to use them for an objective decision about the use of a programming language. 
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Landin [24] was the first to propose the development of a formal framework for 
comparing programming languages. He studied the relationship between program- 
ming languages and constructs, and began to classify some as "essential" and some as 
"syntactic sugar." A typical example of an inessential construct in Landin's sense is 
the let-expression in a functional language with first-class procedures. It declares and 
initializes a new, lexically-scoped variable before evaluating an expression. Whether 
it is present or absent is inconsequential for a programmer since 

let x be v in e is expressible as apply (procedure (x) e) v. 

Similarly, few programmers would consider it a loss if a goto-free, Algol-like language 
had a while but not a repeat construct. After all, 

repeat s until e is expressible as s; while ->e do s. 

Others, most notably Reynolds [36, 37] and Steele and Sussman [40], followed 
Landin's example. They introduced the informal notion of the core of a language 
and studied the expressiveness of imperative extensions of higher-order functional 
languages. Steele and Sussman [40:29] summarized the crucial idea behind this kind 
of classification of language features with the remark that a number of program- 
ming constructs are expressible in an applicative notation based on syntactically 
local, structure- and behavior-preserving translations, but that some, notably con- 
trol statements and assignments, involve complex reformulations of large fractions of 
programs. 

In the realm of logic, Kleene anticipated the idea of expressible or eliminable syn- 
tactic symbols in his study of formal systems [21:§74]. Troelstra [42:1.2] resumed this 
work and introduced further refinements and extensions. Roughly, the additional 
symbols of a conservative extension of a core logic are eliminable if there is a trans- 
lation from the extended logic to its core that satisfies a number of conditions. Two 
of these are important for our purposes. First, the mapping is the identity on the 
formulae of the core language and is homomorphic in the logical connectors. Second, 
if a formula is provable in the extension then so is its translation in the core. Clearly, 
these two conditions imply that this translation preserves the structure of formulae 
and removes symbols on a local basis. 

By adapting the ideas about the relationship among formal systems to program- 
ming languages, we obtain a relation that determines whether a programming lan- 
guage can express a programming construct. More precisely, given two universal pro- 
gramming languages that only differ by a set of programming constructs, {ci, . . . , c n } } 
the relation holds if the additional constructs make the larger language more expres- 
sive than the smaller one. Here "more expressive" means that the translation of a 
program with occurrences of one of the constructs c 4 - to the smaller language requires 
a global reorganization of the entire program. A first analysis shows that this measure 
of expressiveness supports many informal judgements in the literature. Moreover, we 
discover that an increase in expressive power comes at the expense of less "intuitive" 
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semantic equivalence relations. We also discuss some attempts at generalizing the 
measure to a comparison relation for arbitrary programming languages. 

The next section briefly reviews the logical notions of eliminable symbols and def- 
initional extensions. In subsequent sections, we propose a formal model of express- 
ibility and expressiveness along the lines of logical expressiveness, investigate some of 
its properties, and analyze the expressive powers of several extensions of functional 
languages. More specifically, we introduce our formal framework of expressiveness 
based on the notion of expressibility. We demonstrate the abstract concepts by prov- 
ing some sample theorems about A-calculus-based languages as well as a number of 
meta-theorems. Next, we study the expressiveness of an idealized version of Scheme 
and verify the informal expressiveness philosophy behind its design [41]. Following 
this analysis, we briefly speculate how the use of a more expressive language increases 
programming convenience. Finally, we compare our ideas to related work and address 
some open questions. 

2 Eliminable Symbols and Definitional Extensions 

The theory of comparing formal systems is a peripheral topic in logical studies and 
finds little or no space in most textbooks. The following short overview summarizes 
and adapts Troelstra's [42:1.2] descriptions of Kleene's work [21]. 

A formal system is a triple of sets: expressions, formulae, and theorems. The 
second is a subset of the first, the third a subset of the second. Expressions are freely 
generated (in the sense of a term algebra) from a number of non-logical and logical 
operators, e.g., A, — >, etc. The set of formulae is a recursive subset of the set of 
expressions and satisfies certain well-formedness criteria. The set of theorems is the 
subset of the formulae that the formal system defines to be true. If £ is a formal 
system, then Exp(£) is its set of expressions, Fm(£) the set of formulae, and Thm(£) 
the set of theorems; C h t also means t is a theorem of C 

A conservative extension £ of a formal system C is a formal system whose ex- 
pressions are a superset of the expressions over generated from a richer set of 
operators, and whose formulae and theorems restricted to the expressions of CJ are 
the formulae and theorems of C : 

Fm(£) n Exp(£') = Fm(£'); Thm(£) n Exp(£') = Thm(£')- 

A conservative extension £ is a definitional extension of C if there is a mapping 
(p : Exp(£) — > Exp(£') that satisfies the following conditions: 

Fl if(f) e Fm(£') for each / £ Fm(£); 

F2 <p(f) = f for all / £ Fm(£')i 

F3 (p is homomorphic in all logical operators; 
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F4 C h t if and only if CJ V (fi(t); and 



F5 C h t <-> 

Kleene referred to those symbols that generate the additional expressions of the ex- 
tended formal system as eliminable. 

Remark 1 (Weak Expressibility). Kleene's original definition contains a weaker 
version of Condition 3, namely, 

F4' if C h t then £ h <p(t). 

Based on condition F5, it is possible to show that the two definitions are equivalent, 
assuming the usual axioms for <->■ [21: §74]. As we shall discuss in several remarks be- 
low, this is not the case in the context of programming languages. Instead, condition 
F4' leads to a different but related notion of language expressiveness, i 

3 A Formal Theory of Expressiveness 

As a first step towards a formal theory of expressiveness for programming languages, 
we adapt the logical theory of eliminable symbols to the programming language con- 
text. We develop the idea of a programming language as a formal system and re- 
interpret the concepts of conservative extension and eliminability accordingly. Since 
many of the examples in the work of Landin, Reynolds, Steele, and Sussman pre- 
serve not only the global structure of the program but also the local structure of the 
transformed phrases, we consider a stricter notion of eliminability as a second step. 
We refer to this second notion as macro expressibility. It satisfies the additional 
constraint that the transformation of eliminated phrases is always compositional. In 
the two subsections on the respective topics, we prove theorems about the eliminabil- 
ity and non-eliminability of programming constructs and apply them to a simplistic 
prototype language based on the A-calculus. Both notions of expressibility suggest 
natural comparative measures of the expressive power of programming languages, 
which we present in the third subsection. 

3.1 Expressibility 

Like a formal system, a programming language is a system of subsets of a general 
language. More precisely, a programming language is a set of phrases, a subset of 
programs, and a semantics that determines some aspects of the behavior of programs. 

Definition 3.1. (Programming Language) A programming language C consists of 
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• a set of £-phrases, which is a set of freely generated abstract syntax trees (or 
terms), based on a possibly infinite 1 number of function symbols F, Fi, . . . with 
arities a, ai, . . .; 

• a set of ^-programs, which is a non-empty, recursive subset of the set of phrases; 
and 

• a semantics, evalc } which is a recursively enumerable predicate on the set of 
/^-programs. If evalc holds for a program P, the program terminates. 

The function symbols are referred to as programming constructs or programming fa- 
cilities. 

Definition 3.1 is an abstraction of the typical specifications of many realistic pro- 
gramming languages. Most languages have a context-free syntax yet enforce addi- 
tional context-sensitive constraints by recursive 2 decision procedures. Examples of 
such constraints are scoping 3 and typing rules, which ensure that names only occur 
in certain pieces of text and only range over a restricted set of values. 

To avoid restrictive assumptions about the set of programming languages, the 
definition only requires that the semantics observe the termination behavior of pro- 
grams. By omitting any references to the characteristics of results, it is possible to 
consider programming languages with and without observable data. For program- 
ming languages with simple output data, i.e., constants or opaque representations of 
procedures, the definition is in many cases equivalent to a definition that refers to 
the observable output of a program. For a consideration of languages with infinite 
output, e.g., through imperative output statements or through potentially infinite 
lists, the definition needs some adjustments. 

Finally, the above definition of a programming language also shows that, in a 
certain sense, a programming language is a formal system. The set of phrases cor- 
responds to the expressions of a formal system, the set of programs plays the role 
of the set of formulae, and the set of terminating programs is the analog of the set 
of theorems. In the terminology of universal algebra, the set of expressions is the 
universe of a free term algebra [5]; instead of relying on the more typical algebraic 

1 We assume that there is enough structure on an infinite set of constructors for specifying the 
decidability of predicates and the recursiveness of translations on the set of phrases. In the following 
examples, this is obviously the case. 

2 A notable exception is Scheme as defined in the standard report [35], which only has a recursively 
enumerable set of programs: An expression is a Scheme program if and only if it has the same result 
for all possible evaluation orders in its applications. We consider this an unfortunate aberration 
rather than an interesting extension of our definition. 

3 Although most languages impose lexical scoping, Definition 3.1 only accounts for this fact 
through the recursive selection of programs from the set of phrases. An explicit inclusion of the 
lexical scoping structure through a Church encoding [7] of the language in a typed lambda calculus 
is a feasible and interesting alternative but would probably lead to a slightly different definition of 
expressibility and expressiveness. 
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Phrases 

e ::= x \ v | (ee) (expressions) 
v ::= (XyX.e) \ (X n x.e) (values) 

Programs 

e is a program if and only if fv(e) = 0 
where fv(e) is the set of free variables in e 
Semantics 

eval\(e) holds if and only if e — >* v for some v 
Evaluation Contexts 

E::=a\ ((\ v x.e)E) \ (Ee) 

Reduction Steps 

E((X n x.e)e') E(e[x/e']) (fi) 

E((\ v x.e)v) E(e[x/v}) (fi v ) 

where e[x/e'] is the capture-free substituion of e' for all free x in e 
Figure 1: The programming language A 



approach of equational restrictions, the definition uses arbitrary recursive predicates 
for filtering out the interesting subset of programs. Unlike logic, the programming 
language world does not know such ubiquitous constructs as the logical connectors. 

Our prototypical example of a programming language is a derivative of the lan- 
guage A of the pure A-calculus [3]. Figure 1 summarizes its (concrete) syntax and 
semantics. In order to compare the expressiveness of call-by- value and call-by-name 
procedures later in this section, we extend A with a new constructor, A„, and rename 
A to X n . More specifically, the A-phrases are generated from a set of variables (Cl- 
ary constructors), {x,y,z, . . .}, and two families of unary constructors, one for each 

variable x: X v x : term > term (call-by- value abstraction), X n x : term > term 

(call-by-name abstraction), and one binary constructor: • : term X term — > term 
(juxtaposition). Below, A„ and X n denote the sets of all A-constructors. For readabil- 
ity, we use concrete syntax for A-terms and adopt the traditional A-calculus conven- 
tions about its use [3]. 

The constructors X v x and X n x bind the variable x in their term arguments. The 
set of free variables in an expression e, fv(e), is the set of variables that are in e and 
are not bound. If all variables in a A-term are bound, the term is closed. The set 
of A-programs is the set of closed phrases, i.e., there is only one recursive constraint 
that distinguishes programs from arbitrary phrases. 
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The operational semantics of A reflects the semantics of realistic programming 
languages like ISWIM, ML, and Scheme [34]. 4 The specification of the semantics in 
Figure 1 follows the style of extensible operational semantics [8, 10], which is easily 
adaptable to the imperative extensions of A in the following section. An evaluation is a 
sequence of reduction steps on programs according to the normal-order strategy. If the 
program is a value (an abstraction), the evaluation stops. Otherwise, the reduction 
function (uniquely) decomposes the program into an evaluation context, a term with 
a hole (a), and a redex, the contents of the hole. A redex is either an application of a 
call-by-name abstraction to an arbitrary expression (/3) or an application of a call-by- 
value abstraction to a value (f3 v ). In either case, the reduction function replaces the 
redex, ((X x .b)a) } by a new version of the procedure body, b[x/a]. Then the evaluation 
process starts over again. 

Summarizing the standard reduction process as a predicate on programs yields the 
operational semantics of A. Some useful examples of phrases are the call-by-name 
and call-by- value fixed point operators, which facilitate the recursive definition of 
functions: 

Y n = (X n f.(X n x.f(xx))(X n x.f(xx))) 

and 

Y„ = (X v fx.(X v g.gg)(X v x.f(X v x.(gg)x))x). 

Two simple diverging programs are fi n = Y n (X n x.x) and fi„ = Y v (X v xy .xy)(X v x.x). 

To illustrate the impact of syntactic constraints on programs, we also define A*, 
a typed variant of A. A* has the same set of phrases as A but uses a type checking 
algorithm for filtering out valid programs. A A* program is not only closed but is 
also typable as either an integer or a higher-order functional on integers according 
to the type inference system in Figure 2. It easily follows from Milner's [27] initial 
work on polymorphism that typability is a recursive predicate for A*. The semantics 
of A*-programs is the same as that of their untyped counterparts. 

A* is a typical example of a monomorphic language: all occurrences of a A-bound 
variable have the same type. As a consequnce, the typing constraints of A* exclude 
typical A-programs like Xx.(xx) or phrases like (xx) in programs. Indeed, the Y 
operator for defining recursive functions must be typed explicitly because it would 
not pass the other type rules. 

Based on the interpretation of a programming language as a formal system, it is 
easy to define the notion of a conservative programming language extension. 

Definition 3.2. (Conservative Extension & Restriction) A programming language 
£ is a conservative extension of a language C if 

• the constructors of CJ are a subset of the constructors of C with the difference 
being {Fi, . . . , F n , . . .}, which are not constructors of 

4 As usual, this operational semantics has only remote connections to the equational theory of 
the A-calculus. 
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Programs 

e is a program if and only if it is closed and 0 h e : r 

Types 

t ::= l | t — > t 

Type Assertion 

A: Variables -e-+ Types (finite functions) 
where A[x/t](x) = t A[x/r](y) = A(y) (functional update) 
Type Inference 

A[x/t] he:r' 



A h x : t if A(x) = t 



A h A„ , n x.e : r — > t' A\- (ee r ) : r 



-4 I" Y n : (r — ► r ) — > t AhY„: ((r r') (r r')) (r r') 

Figure 2: A* 



• the set of £'-phrases is the full subset of £-phrases that do not contain any 
constructs in {Fi, . . . , F n , . . .}; 

• the set of ^'-programs is the full subset of ^-programs that do not contain any 
constructs in {Fi, . . . , F n , . . .}; and 

• the semantics of eval c i } is a restriction of £'s semantics, i.e., for all C- 
programs P, eval c i(P) holds if and only if evalc(P) holds. 

Conversely, CJ is a conservative restriction of C 

To emphasize the constructors on which the restriction and extension differ, we 
write C = C \ {Fi, . . . , F n , . . .} and C = £ + {Fi, . . . , F n , . . .}. We also use the 
notation to denote the natural restriction and extension that result from subtracting 
or adding facilities to the syntax (provided the respective languages and, in the latter 
case, a semantic specification exist). 

In our running example, the restricted language A n is A without A„-abstractions, 
A„ is A without call-by-name abstractions: 

A n = A \ A„; A v = A\\ n . 

A restriction of the above evaluation process to A„- and A n -phrases yields a call-by- 
value and a call-by-name semantics, respectively. The corresponding sublanguages of 
A* are defined similarly. 
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To enrich the set of examples, we add a let construct to A. There is one binary 
let constructor for each variable x: \et x : term X term — > term. Like Xx } \et x binds 
its variable, namely in the second subexpression. The concrete syntax for \et x (e } e') 
is 

(let x be e in e'). 

The semantics of A + {let^}, or A + let for short, requires an additional clause in the 
specification of evaluation contexts 

E ::= ... | (let x be E in e) 

and an additional clause for the reduction function: 

£"(let x be v in e) — > E(e[x/v]). (let„) 

Otherwise the definition of the semantic predicate in Figure 1 stays the same. It is 
trivial to check that A + let is a conservative extension of A. 

The extension of A* with let additionally requires a type inference rule for the new 
construct. For greater flexibility, the new rule only requires that, at each occurrence 
of the abstracted variable, the named subexpression is typable with some type: 

Ahe:r; A h e[x/e'] : t' 
A h (let x be e in e') : r' 

A* + let is polymorphic in the spirit of ML [27, 28, 43:43, 44]. Unlike A-bound vari- 
ables, let-bound variables can have several types. For example, x in (let x be (Xy.y) in (xx)) 

conceptually assumes two different types: (t > t) > (t — > t) and (t > i), which 

makes the expression a legal program despite the self-application of a variable. Again, 
it is easy to see that the extension is conservative with respect to A*. 

For the re-interpretation of the logical notion of eliminability, we need to be more 
flexible. The non-existence of ubiquitous programming language features in the sense 
of logical connectors raises the question whether the mapping from the extended lan- 
guage to the core should be homomorphic and, if so, on which set of features. At 
this point, we recall the above-mentioned desire that our translations be structure- 
preserving, and the idea that the homomorphic character of a translation naturally 
corresponds to this property. To preserve the structure of programs as much as pos- 
sible, we require that the translations be homomorphic in all programming facilities 
of the core language. 

Definition 3.3. (Eliminability; Expressible Programming Constructs) Let C he & 
programming language and let {Fi, . . . , F n , . . .} be a subset of its constructors such 
that C = £\{Fi, . . . , F n , . . .} is a conservative restriction. The programming facilities 
Fi, . . . , F n , . . . are eliminable if there is a recursive mapping <p from £-phrases to C - 
phrases that satisfies the following conditions: 
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El <p(e) is an ^'-program for all ^-programs e; 

E2 <£>(F(ei, . . . , e a )) = F(t^(ei), . . . , <p(e a )) for all facilities F of CJ , i.e., (/? is homo- 
morphic in all constructs of C!\ and 

E3 evalc(e) holds if and only if eval £/(<£>(e)) holds for all ^-programs e. 

We also say that £ can express the facilities Fi, . . . , F n , . . . with respect to C We 
omit the qualification if the language universe is clear from the context. By abuse of 
notation, we write <p : C > £ . 

Condition E2 in this definition implies that the mapping <p is the identity on the 
language £ . It corresponds to conditions F2 and F3 of the logical notion of elim- 
inability. According to the above interpretation of a programming language as a 
formal system, an interpretation of condition F4 in Section 2 requires that the trans- 
lation of a terminating program in the extended language is a terminating program 
in the restricted language. This is precisely the contents of Condition E3. Finally, 
the last condition of the logical notion of eliminability, F5, has no counterpart in 
a programming language context because of the lack of a ubiquitous programming 
construct. 

Remark 2 (Weak Expressibility). By using an adaptation of Kleene's original 
Condition F4' (see Remark 1 in the previous section) instead of the third condition in 
the preceding definition, we get a weak notion of expressibility. The revised condition 
takes on the following shape: 

F3'. if evalc(e) holds then eval £/(<£>(e)) holds, 

The intuition behind this definition is that the translated phrase has at least as many 
capabilities as the original one. The terminology reflects our belief that any differ- 
ences in behavior should be noted as a failure of complete expressibility. Moreover, 
the terminology is also consistent with the fact that expressibility implies weak ex- 
pressibility. i 

An alternative understanding of the above definition is that the translation maps 
phrases constructed from eliminated symbols to observationally indistinguishable 
phrases in the smaller language. In other words, replacing the original phrase with 
its translation does not affect the termination behavior of the surrounding programs. 
This relation between two phrases of programming languages is widely studied in 
semantics and is known as operational (or observational) equivalence [25, 29, 33, 34]. 
After developing the formal definition of operational equivalence, we can characterize 
sufficient conditions for the eliminability of programming constructs. 

A formal definition of the operational equivalence relation relies on the auxiliary 
notion of a program context. 
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Definition 3.4. (Contexts; Program Contexts) An n-aiy context over£, C(«i, . . . ,a n ), 
is a freely generated tree based on £'s constructors and the additional, 0-ary construc- 
tors a n , called meta-variables. All subtrees of C(«i, . . . , tt n ) are also n-ary 
contexts. If C(«i, . . . , ct n ) is a context and ei, . . . , e n are phrases in £, then the m- 
stance C(ei, . . . , e n ) is a phrase in £ that is like C(«i, . . . , tt n ) except at occurrences 
of a.i where it contains the phrase e 4 -: 

• if C («i, . . . , ct n ) = cti then C (ei, . . . , e n ) = e 4 -, and 

• if C(«i, . . . , ct n ) = F(Ci(ai, . . . , tt n ), . . . , C a (o!i, . . . , for some F with arity 
a then C(e 1 , ...,e n ) = F(Ci(ei, . . . , e n ), C a (e 1 , e n )). 

An C-program context for a phrase e is a unary context, C(a), such that C(e) is 
a program. 

For A n , the context Co(ct) = (X n xy .ct)(\ n x.x)ft n is a program context for all 
expressions whose free variables are among x and y. 

Since the semantic predicate of a programming language only tests a program 
for its termination behavior, our definition of operational equivalence compares the 
termination behavior of programs. 5 

Definition 3.5. (Operational Equivalence) Let C he & programming language and 
let evalc be its operational semantics. The £-phrases e\ and e 2 are operationally 
equivalent, e\ =c e 2 , if there are contexts that are program contexts for both e\ and 
e 2 , and if for all such contexts, C(a), eval c(C (e\j) holds if and only if eval c(C (e 2 )) 
holds. 

With the above program context Co, it is possible, for example, to differentiate the 
phrases x and y. Since fi n diverges, Cq(x) = (\ n xy .x)(\ n x .x)ft n terminates whereas 
C 0 (y) = (\ n xy.y)(\ n x.x)rL n diverges. 

Figure 3 contains a sequent calculus of operational equivalence on A„ + let, which 
is used below in Proposition 3.7; the proof system is similar to Riecke's for a typed 
version of A„ [38]. The calculus proves equations over the language from premisses 
(T) that are finite sets of equations. It is sound but incomplete, i.e., if 0 h e = e 1 then 
e = e 1 but not vice versa. 

Remark 3 (Weak Expressibility). A replacement of the "if-and-only-if" con- 
dition with a simple "if" condition yields the notion of operational approximation, 
which plays the same role for weak expressibility as operational equivalence for ex- 
pressibility. More specifically, the term e\ operationally approximates e 2 , e\ £ £ e 2 , 
if for all program contexts, C(a), for e\ and e 2 , evalc(C(e 2 )) holds if evalc(C(ei)) 
holds. — In conjunction with the above context Co, the program context C\(a) = 
(\ n y x .a)(\ n x .x)ft n shows that x and y do not approximate each other, i 

5 In many cases, our definition is equivalent to the more traditional definition that compares the 
termination behavior of programs and the results, provided they are among a set of observable data. 
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r h (X v x.e)v = e[x/v] 



rhfie = Sl 



r h eft = 



T h e = e' 
r h A„a;.e = A, 



Mr) 



rhe = e' 
r h ee" = e'e" 



rhe = e' 
r h e"e = e"e' 



r h (let j; be b in e) = e[x/v] T h (let a; be 0 in e) = 0 



r h e = e' 

r h (let a; be e in e") = (let x be e' in e") T V ; 



, , r U {e = fi} h ei = r U {e = f } h ei = e 2 for all v 

ru{e = e}he = e — 

1 h ei = e 2 

r h e = e' T h e = e'; lhe' = e" 



r h e = e 



r h e' = e r h e = e" 

Figure 3: A Calculus for Operational Equivalence on A„ + let 



We now have everything in place to formalize our above idea about the connection 
between eliminable programming constructs and their translations. The following the- 
orem shows that, at least to some extent, the elimination of expressible programming 
constructs from a program is a local process and keeps the program structure intact. 

Theorem 3.6 Let C = £ + {Fi, . . . , F n , . . .} be a conservative extension of CJ . If 

(p : C > £ is homomorphic in all facilities of CJ and preserves program-ness, and if 

F 4 -(ei, . . . , e ai ) =c <£>(F 4 -(ei, . . . , e aj )) for all F 4 - and all C-expressions ei, . . . , e ai , then 
£ can express the facilities Fi, . . . , F n , . . .. 

Proof. It suffices to show that condition E3 of Definition 3.3 holds for <p. Assume 
that P is an ^-program such that evalc(P) holds. By the construction of P, there is 
a context C(«i, . . . , ct n ) such that 

P = C{p 1 , . . .,p n ) 

where pi , . . . , p n are the finite number of outermost occurrences of phrases constructed 
from some facilities in Fi, . . . , F n , . . .. Thus, C(cti } p 2} . . . } p n ) is a program context 
for pi. It follows from the theorem's assumption that 

eval c(C(pi } . . . ,p n )) holds if and only if evalc(C((p(pi) } . . . ,p n )) holds. 



12 



Repeating this step n times proves that 

eval c(C(pi } . . . ,p n )) holds il and only if evalc(C((p(pi) } . . . , <p(p n ))) holds. 
But, since C does not contain any facilities in Fi, . . . , F n , . . ., 

C(p(pi), . . .,<p(p n )) = <f(C(pi, . . .,p n )) = <f(P). 
Moreover, since C conservatively extends 

evalc(P) holds if and only if eval £'(<p(P)) holds. 
This completes the proof, i 

Remark 4 (Weak Expressibility). The theorem holds for weak expressibility even 
if we replace operational equivalence by operational approximation: If F 4 -(ei, . . . , e ai ) £ 
<£>(F 4 -(ei, . . . , e aj )), then the Fi, . . . , F n , . . . are weakly eliminable. i 

An application of this theorem shows that let is an example of an eliminable con- 
struct. For the typed setting, this provides a precise formalization of the folk theorem 
that ML-style polymorphism is expressible in a monomorphic language. 6 The two ex- 
amples also reveal a striking difference between the typed and the untyped language 
variant. Whereas the untyped let expression simply abbreviates an application as 
illustrated in the introduction, its typed counterpart maps to a version of the let 
body in which each occurrence of the abstracted variable is substituted with a copy 
of the named expression. The reason for this difference is that in the typed case the 
translation must not only preserve the semantics but also the typability in order to 
preserve program-ness. 

Proposition 3.7 The constructor let is eliminable in call-by-value languages, 
(i) A v can express let with respect to A v + let. 
(ii) A* can express let with respect to A* + let. 

Proof. (?) Set ip(\et x be e in e') = (\ v x.<p(e'))<p(e). To show that the two phrases 
are operationally equivalent we proceed by induction on the number of let's in the 
subexpression. The b for e, e' in A„ proceeds as follows. By the homomor- 

phism constraint, f(e) = e and f(e') = e'. By the laws in Figure 3, 

e = 0 h (let x be e in e') = 0 = (\x.e')e 

and, for all values v, 

e = v \~ (let x be e in e') = e'[x/v] = (Ax.e')e, 
6 An alternative approach to a formalization of this folk theorem is due to Wand [44]. 
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and therefore, 

(let x be e in e') =A+let (Ax.e')e. 

The induction step proceeds along the same line. 

(m) Set <p(\et x be e in e') = ((\ v d.(p(e')[x / \p(e)])(p(e)) where d does not occur 
free in e or e' . To verify that this translation maps programs to programs, it suffices 
to prove that the translation preserves the implicit type assignment. We proceed by 
induction on the number of let-expressions in the program and show that, given a 
fixed set of type assumptions, f(e) has the same type as e. Assume e and e 1 are 
let-free, and consider the typing of an instance of let: 

A h (let x be e in e') : r' 

for some A and r'. By the setup of the inference system, A h e : r for some r and 
A h e'[x/e] : r'. Since, by assumption, f(e) = e and f(e') = e', A h <£>(e) : r and 
A h ( J c(e / )[x/( J c(e)] : r'. The rest follows easily: A h (A„ J.( J c(e)[x/( J c(e')]) : r — )■ r' and 
also 

A^ (X v d.if(e)[x/if(e')])if(e):r'. 

The induction step requires a lemma that proves <p(e)[x / \p(e')] has the same type as 
e[x/e'] if <p(e) and f(e') have the same type as e and e'. 

The proof that preserves the semantics of typed let-programs follows the same 
pattern as part (i). Observe that 

e = 0 h (let x be e in e') = 0 = (AJ.e'[x/e])0 = (Ad.e'[a;/e])e 

and also, for all v, 

e = v \~ (let x be e in e') = e'[x/v] = (\d.e'[x/v])v = (\d.e'[x / e])e. 

The rest is obvious, i 

The converse of Theorem 3.6 does not hold. That is, the facilities Fi, . . . , F n , . . . 
may still be expressible in CJ even though the translation from C to C maps an 
eliminable phrase to an observably distinct element. One reason for this is that the 
set of programs may not contain an element such that F 4 -(ei, . . . , e ai ) for some F; 
occurs in a context over the restricted language, in which case it is irrelevant how the 
mapping <p translates this phrase. Thus, by imposing an appropriate condition, we 
can get a theorem on the non-expressibility of programming constructs. 

Theorem 3.8 Let C = £ + {Fi, . . . , F n , . . .} be a conservative extension of C . 

If for all mappings <p : C > £ that are homomorphic in all facilities of £ , 

F 8 (ei, . . . , e ai ) ¥c ^(F;(ei, . . . , e ai )) for some C-expressions e 1} . . . , e at and F t in Fi, . . . , 
and if there is a context C(a) over £ that witnesses this inequality, then CJ cannot 
express the facilities Fi, . . . , F n , . . .. 
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Proof. Let <p : C — > £ be an arbitrary mapping that is homomorphic in all facilities 
of C . Suppose <£>(F 4 -(ei, . . . , e aj )) = e and let C(a) be the context over C that observes 
the operational difference between e and F 4 -(ei, . . . , e ai ). Since <p is homomorphic over 

<^(C(F,-( ei , . . . , e ai ))) = C(^(F 8 ( ei , . . . , e ai ))) = C(e). 

But, by assumption, evalc(C (F 4 -(ei, . . . , e aj ))) holds while evalc(C (e)) and eval ^(C (e)) 
do not hold (or vice versa). This implies that no mapping that is homomorphic over 
£ can possibly satisfy condition E3 of Definition 3.3 if the antecedent of the theorem 
holds, and that consequently the programming constructs Fi, . . . , F n , . . . cannot be 
eliminable. i 

Based on this first non-expressibility theorem, we can now prove that call-by-name 
A cannot express call-by- value abstractions and vice versa. 

Proposition 3.9 A extends both A v and A n . 
(i) A n cannot express X v with respect to A. 
(m) A„ cannot express X n with respect to A. 

Proof. 7 (i) According to the semantics for A n , the application of an abstraction 
(X n y.p) to an argument can only proceed in one of the following three manners: 

1. it may uniformly diverge for all arguments: whenever (X n y.p)e — >* e 1 there is 
always an e" such that e 1 — > e"; 

2. it may uniformly converge to a value for all arguments, including fi n : for all 
expressions e there is a value v e such that (X n y.p)e — >* v e ; 

3. it may activate the argument for a first time: (X n y.p)e >* ee\ . . . for some 

ei, . . . , e k . 

The proof of this auxiliary claim is a simple induction on the length of the reduction 
sequence. Also, it is easy to check that whenever e — >* e 1 then E(e) — >* E(e'). 

Let (X v x.e) be an abstraction in A that converges upon application to some val- 
ues. More specifically, let e and v be in A n and assume that for some value it, 
(X v x.e)v(X n x.x) — >* u. Then we claim that C(a) = (a v)(X n x.x) is the context that 
we are looking for in order to apply Theorem 3.8. 

Assume that <p : A > A n is a structure-preserving translation. Then (p(X x v.e) is 

or reduces to a value in A n . Let X n y.p be this value. Since the original abstraction 
terminates upon application to some value, the translation of this application must 
terminate as well. Therefore X n y.p cannot diverge uniformly. On the other hand, 
the pre-image, X v x.e } also diverges upon application to fi n , which implies that the 

7 Improved by Carolyn Talcott. 
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translation cannot converge uniformly. Thus, let ei,...,efc be the expressions such 
that 

C((fi(X v x.e)) = (fi(X v x.e)v(X n x.x) >* (X n y.p)v(X n x.x) — >* ve x ... e k (X n x.x). 

Setting e = X n x.x and v = X n x.ft n (which satisfy the original asumptions), it is trivial 
to see that this program now diverges. If n = 0 the reduction continues with: 

. . . > (X n x .Q n ^}(X n x .x^J > O n > . . . , 

otherwise, it is: 

. . . — > (A n x.O n )ei . . . e k (X n x.x) — > O n e 2 . . . e k (X n x.x) — > . . . 

That is, whereas C(X v x.e) converges to (X n x.x) } C(tp(X v x.e)) diverges. By the 
preceding theorem, we have shown our claim. 

(m) This part is much simpler. Take C(a) = (aO„), e = (X n x.(X n x.x)) and 
assume that <p is a structure-preserving translation. Clearly, eval(C(X n x.(X n x.x))) 
holds, but, C((p((X n x.(X n x.x)))) diverges. Hence, a structure-preserving translation 
cannot preserve operational equivalence, which proves the claim, i 

Remark 5 (Weak Expressibility). By Remark 4, A n can weakly express A„ 
because X v x.e £ A X n x.e. i 

An immediate corollary of Theorem 3.8 is that if for some phrase with an elim- 
inable symbol there is no operationally indistinguishable expression in the restricted 
language, then the restricted language is less expressive than the full language. We 
use this corollary instead of the full theorem in Section 4.2. 

Corollary 3.10 Let C = £ + {Fi, . . . , F n , . . .} be a conservative extension of CJ . If 
for some F 4 -(ei, . . . , e ai ) there is a program context C over C but there is no e in C 
such that F 4 -(ei, . . . , e ai ) =c e, then C cannot express the facilities Fi, . . . , F n , . . . 

3.2 Macro Expressibility 

Although the definition of eliminable programming construct is a satisfactory first step 
towards a better understanding of the formal structure of programming languages, it 
does not completely account for the idealized notion of "syntactic sugar" of Landin 
and others [24, 36, 37, 40] as discussed in the introduction. In many cases, the 
elimination of "syntactic sugar" constructs not only preserves the global program 
structure but also the structure of the subexpressions of phrases built from eliminable 
constructs. 

Recall the two examples from the introduction: 
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1. In a functional language with first-class functions, a let expression is simply 
an abbreviation of the immediate application of an anonymous procedure to an 
argument: 

let x be v in e is expressible as apply (procedure (x) e) v. 

2. In a goto-free, Algol-like language, 

repeat s until e is expressible as s; while ->e do s. 

In both examples, the translation of a composite phrase is the (fixed) composition 
of the translation of its subphrases. More technically, the translation of a phrase 
is the evaluation of a term (in the sense of universal algebra [5]) over the restricted 
language at the translations of the subphrases. As mentioned above, terms correspond 
to contexts in our framework; for clarity, we refer to contexts as syntactic abstractions 8 
in relation to the following definition and its uses. 

Definition 3.11. (Macro Eliminability; Macro Expressibility) Let C he & program- 
ming language and let {Fi, . . . , F n , . . .} be a subset of its constructors such that 
£ = C \ {Fi, . . . , F n , . . .} is a conservative restriction. The programming facilities 
Fi, . . . , F n , . . . are macro eliminable if they are eliminable and if the eliminating map- 
ping (p from C to £ satisfies the following, additional condition: 

E4 For each a-ary construct F £ {Fi, . . . , F n , . . .} there exists an a-ary syntactic 
abstraction, A, over £ such that 

<£>(F(ei, . . . , e a )) = A{fi(e 1 ), . . . , fi{e a )). 

We also say that £ can macro-express the facilities Fi, . . . , F n , . . . with respect to £ 

Remark 6 (Weak Expressibility). If some facilities are weakly expressible and 
satisfy the additional condition, we call them weakly macro-expressible, i 

Since macro expressibility is a restriction of simple expressibility, Theorem 3.6 on 
the eliminability of constructs requires some adaptation. 

Theorem 3.12 Let C = £ + {Fi, . . . , F n , . . .} be a conservative extension of £ . 

If fi : C > £ is homomorphic in all facilities of CJ and preserves program-ness, 

and if there is a syntactic abstraction A{ for each F 4 - in Fi, . . . , F n , . . . such that 
F 8 (ei, . . . , e ai ) = c A t (fi(e 1 ) } . . . , fi{e ai )) for all C-expressions e 1} . . . , e at , then £ can 
macro-express the facilities Fi, . . . , F n , .... 

8 In Lisp-like languages, syntactic abstractions are realized as macros [22]; logical frameworks 
know them as notational abbreviations [17]. The terminology of equational algebraic specifications 
[16] refers to syntactic abstractions as derived operators. 
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Proof. It is easy to see that the additional condition in the antecedent is precisely 
what is needed to adapt the proof of Theorem 3.6 to the stronger conclusion, i 

Moreover, the additional condition E4 permits a simplification of the theorem to 
a corollary that no longer makes any reference to the translating map. The corollary 
is used in Section 4. 

Corollary 3.13 Let C = £ + {Fi, . . . , F n , . . .} be a conservative extension of £ . If 
there is a syntactic abstraction A{ for each F 4 - in Fi, . . . , F n , ... so that F 4 -(ei, . . . , e ai ) =c 
A 8 -(ei, . . . , e ai ) for all C- expressions ei, . . . , e ai , then £ can macro-express the facilities 
Fi, . . . , F n , . . .. 

By Proposition 3.7, A„ can express let. A simple check of the proof reveals that 
the translation between the two languages satisfies the antecedent of the corollary, 
and that therefore let is also macro-expressible. More importantly, however, the 
additional condition E4 in Definition 3.11 also leads to a stronger meta-theorem on 
the Tioft-expressibility of facilities. The new theorem shows that new programming 
constructs add to the expressive power of a language if their addition affects existing 
operational equivalences. 

Theorem 3.14 Let C\ = Co + {Fi, . . . , F n , . . .} be a conservative extension of Co- 
Let =o and =i be the operational equivalence relations of Co and C\, respectively. 

(i) If the operational equivalence relation of C\ restricted to Co expressions is not 
equal to the operational equivalence relation of Co, i.e., = 0 7^ (— i \Co), then Co 
cannot macro-express the facilities Fi, . . . , F n , . . .. 9 

(m) The converse of (i) does not hold. That is, there are cases where Co cannot 
express some facilities Fi, . . . , F n , . . ., even though the operational equivalence 
relation of C\ restricted to Co is identical to the operational equivalence relation 
of C 0 , i.e., = 0 = (=1 \C 0 ). 

Proof. Let eval 0 and evali be the respective evaluation predicates for Co and C\. 

(i) A difference between the restricted operational equivalence relation of C\ and 
that of Co implies that there are two phrases e and e 1 in Co and C\ such that either 
e = 0 e 1 and e ^ e 1 or e ^ 0 e 1 and e =1 e'. For the first case, let C(a) be a context 
over C\ that can differentiate the two phrases e and e'. Let us say, without loss of 
generality, that 

evali holds for C(e) but not for C(e'). 

Now, assume contrary to the claim in the theorem that Co can express the facilities 
Fi, . . . , F n , . . .. Then, there is a mapping <p : C\ — > Co that satisfies conditions El 

9 An extension of an equivalence relation to a larger language is also called conservative if the 
restriction to the old syntax yields the original equivalence relation. To avoid confusion, we will not 
use this terminology here. 
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through E4 of Definitions 3.3 and 3.11. By conditions El and E3, the programs 
C(e) and C(e') have counterparts in Co, <p(C(e)) and t^(C(e')), that have the same 
termination behavior: 

eval 0 ((p(C(e))) holds because eval 1 {C{e)) holds 

and 

eval 0 ((fi(C(e'))) does not hold because eval 1 {C{e')) does not hold. (f) 

By conditions E2 and E4, the programs (p(C(e)) and (p(C(e')) can only differ in a 
finite number of occurrences of e and e' . In other words, there is a program context 
D(a) over Co such that (p(C(e)) = D(e) and (p(C(e')) = D(e'): for the proof, see 
the Translation Lemma below. Next, from the assumption e = 0 e', it follows that 
(p(C(e)) = D(e) and (p(C(e')) = D(e') have the same termination behavior in Co, i.e., 

eval 0 (D(e)) holds if and only if eval 0 (D(e')) holds. 

But this contradicts the above fact (f) that (p(C(e)) = D(e) converges and (p(C(e')) = 
D(e') diverges, which concludes the first case. 

For the second case, assume e ^ 0 e 1 and e =1 e'. This assumption actually implies 
that 

there are no contexts over Co that complete both e and e 1 to programs. (*) 

For otherwise, there must be a context C(a) such that eval 0 (C(e)) holds while 
eval 0 (C(e')) does not. Since C\ is a conservative extension of Co, eval 1 {C{e)) holds, 
evali(C(e')) does not, and therefore contrary to the assumption, e ^ x e'. 

Now again, assume contrary to the claim of the theorem that Co can express the 

additional facilities in C\ via an appropriate translation <p : C\ > Co- Since e and 

e 1 are operationally equivalent in C\, there must be a context C(a) over C\ such that 
evali(C(e)) and eval 1 {C{e')). By assumption, eval 0 ((fi(C(e))) and evalo((fi(C(e'))). 
Again by the Translation Lemma, the two translated programs are instances of the 
same program context D(a) such that (p(C(e)) = D(e) and (p(C(e')) = D(e'). But 
by the above fact (*), such a context cannot exist, and we have thereby arrived at a 
contradiction. This concludes the second case of claim (i). 

To finish the proof of claim (i), we must finally show that a homomorphic function 
preserves the structure of a program. 

Translation Lemma. Let <p : C\ > Co be a translation that satisfies Conditions (0) 

through (3) in Definitions 3.3 and 3.11. Let C(a) be a context over C\. Then, there 
is a context D(a) over Co such that (p(C(e)) = D(e) and (p(C(e')) = D(e'). 
Proof. The proof is an induction on the structure of C(a). The only interesting 
case is the following. Say, C(a) = F(Ci(a), . . . , C a (a)) for some F in Fi, . . . , F n , . . .. 
Then, 

ip(F(C\(e), . . . , C a (e)) = A(tp(C\(e)), <p(C a (e))) 
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and 

Vifidie'), . . . , C a (e')) = A^C^e')), . . . , <p(C a (e'))) 

for some fixed syntactic abstraction A over Co by condition E4. By inductive hy- 
pothesis, there are contexts for f < i < a, such that (p(Ci(e)) = -D;(e) and 
<p(Ci(e')) = Di(e'). But then = A(Di(a) } . . . , D a (a)) is the context correspond- 
ing to C(a). The other cases are similar but easier. *i emm a 

The proof of the Translation Lemma finishes the proof of case (i). 

(ii) We only sketch the construction of an example that proves claim (ii). For 
another example that is more interesting and fits more smoothly into this paper, see 
Subsection 4.2 on the control structure of Idealized Scheme. 

For the base language, take the simply typed A-calculus with a fixed point oper- 
ator, whose types are either base types or arrow types. Because of the type system, 
it is impossible to define the typical cons, car, and cdr functions for pairs of values 
of arbitrary types. Hence this simply-typed language cannot express these functions. 
On the other hand, also due to the type system of the language, the new functions 
cannot be bound to free variables in phrases of the sublanguage, which implies that 
the additional functions on pairs (of distinctly typed components) cannot be used to 
distinguish phrases in the simply typed language. It follows that pairing functions and 
selectors increase the expressive power without destroying operational equivalences 
of the underlying language, i 

Based on Theorem 3.14, we can show that the sublanguage A„ is not strong 
enough to macro-express call-by-name abstraction, and that A n is not strong enough 
to macro-express call-by- value abstraction. The proofs utilize the first half of the 
proof of claim (i). 

Proposition 3.15 A extends both A v and A n . 

(i) A n cannot macro-express X v with respect to A. 

(ii) A v cannot macro-express X n with respect to A. 

Proof. The claims are obviously consequences of Proposition 3.9. 

(i) A direct proof for the first claim is derived from a theorem by Ong [31: Thm. 4.1.1], 
based on the preceding meta-theorem. Consider the phrases x(X n y .x(YK.)ily)(YK.) 
and i(i(YK)0)(YK). The two are equivalent in an adequate model of A n [31] and 
are therefore operationally equivalent: 

x(\ n y.x(YK)tty)(YK) = An x(i(YK)0)(YK). 

The operational reasoning for a verification of this equivalence is as follows. No matter 
which argument the procedure x evaluates first, the expression (YK) eventually ap- 
pears in the hole of the evaluation context, which leads to an immediate termination 
of the program evaluation. 

10 Gordon Plotkin pointed out Abramsky's [1] and Ong's [31] work on the lazy A-calculus, which 
corrected a mistake in an early draft. 
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In the full language A, the above analysis no longer holds: Call-by- value proce- 
dures can evaluate and discard the expression (YK) in a way that does not affect the 
rest of the program. Thus, the context 

C(a) = (X n x.a)(X v x.(X n y.y)) 

can distinguish between the two phrases: C(x(A n j/.x(YK)Oj/)(YK)) terminates, but 
C(i(i(YK)0)(YK)) diverges. By Theorem 3.14, A n cannot express A„. 

(m) Consider the expressions \ v f.f(\ v x.x)ft and \ v f.ft. In the pure call-by- value 
setting, the two are operationally equivalent: 

\ v f.(f(\ v x.x))tt = Av XJ.tt. 

Both abstractions are values; upon application to an arbitrary value, both of them 
diverge. A formal proof is straightforward, based on the proof rules in Figure 3. In 
the extended language A, however, we can differentiate the two with the context 

C(a) = a(X v x.(X n y.x)). 

The context applies a phrase to a function that returns the value of the first argument 
after absorbing the second argument without evaluating it. Hence, C(\ v f.f(\ v x.x)ft) 
terminates while C(\ v f.Q) diverges, which proves that the extension of A„ to A does 
not preserve the operational equivalence relation. Again by Theorem 3.14, X n is not 
expressible, i 

Remark 7 (Weak Expressibility). Remark 5 and the proof of Proposition 3.15 
show that Theorem 3.14 does not carry over to weak expressibility because A n can 
weakly macro-express A„ and yet = An ^= A . That is, even in the case of a language 
extension that does not preserve the operational equivalence or approximation rela- 
tion, the restricted language may already be able to express the new facilities in a 
weak sense, i 

For a second application of Theorem 3.14, we show that the polymorphic let 
construct of A* is not macro-expressible in A*. On one hand, this lemma confirms the 
folk knowledge that a polymorphic let adds expressive power to a monomorphically 
typed programming language. It does not contradict the above proposition, which 
only shows that a polymorphic let is expressible in a monomorphic language. On 
the other hand, this lemma provides an example of an interesting facility that is 
expressible but not macro-expressible relative to the same language. The proof relies 
on the second part of claim (i) in the meta-theorem. 

Proposition 3.16 A* cannot macro-express let with respect to A* + let. 
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Proof. Consider the expressions ((gx)(f /))) and (X v d.((gx)(f f)))(gx). Since both 
contain a self- application of the variable /, there is no A* program context for the 
two expressions, and the two programs cannot be operationally equivalent: 

£a* (X v d.((gx)(ff)))(gx) 

Still their dynamic behavior is the same. The difference between the two programs 
is that the second computes the application of g to x twice, throwing away the first 
result via a vacuous abstraction: 

gx = 0 h (gx)(ff) = !)(//) = 0 = (\ v d.((gx)(ff)))Sl = (X v d.((gx)(ff)))(gx) 

and, for all values v, 

gx = vh (gx)(ff) = v(ff) = (X v d.((gx)(ff)))v = (X v d.((gx)(ff)))(gx). 

Thus, in the extended language, where the variable / can be let-bound in an appro- 
priate context, the two program fragments are equivalent: 

(Miff)) =A<+let (X v d,((gx)(ff)))(gx). 

Together with the above inequality, this proves the proposition, i 

Propositions 3.15 and 3.16 provide several examples of pairs of universal program- 
ming languages that we can differentiate according to our expressiveness criterion. 
With the full language A, it also provides an example of a language that can express 
more than A„ and A n . We have come to a point where we can formally distinguish 
the expressive power of programming languages. 

3.3 Expressiveness 

The two notions of expressibility are also simple comparison relations for languages 
and their conservative extensions. For a comparison of arbitrary programming lan- 
guages, these relations are too weak. One solution is to conceive of our abstract 
programming languages as signatures (or types in the sense of universal algebra [5]) 
for classes of real programming languages. It is then possible to compare languages 
by comparing their signatures if one signature happens to be a conservative exten- 
sion of the other. Though appealing at first glance, this idea only relaxes syntactic 
constraints such that the languages under comparion do not have to have the same 
syntax. 

An alternative solution is to consider a common language universe that is a con- 
servative extension of two or more programming languages. Given a common universe 
that fixes the meaning of a number of interesting programming constructs, there is 
a natural extension of the notion of expressibility to a notion of relative expressive 
power. Intuitively, a programming language is less expressive than another if the 
latter can express all the facilities the former can express in the language universe. 
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Definition 3.17. ((Macro) Expressiveness) Let Co and C\ be conservative language 
restrictions of C. C\ is at least as (macro-) expressive as Co with respect to C if C\ 
contains or can (macro-) express a set of C- constructs whenever Co contains or can 
(macro-) express the additional facilities. 

The expressiveness relation is obviously a pre-order on sublanguages in a given 
language framework; it is also monotonic in its third argument provided the extension 
to the universe is conservative. 

Theorem 3.18 Let Co, C\, C2 be conservative restrictions of C, and let C be a 

conservative restriction of CJ . 

(i) Co is less expressive than Co with respect to to C. 

(ii) If Co is less expressive than C\ with respect to C and C\ is less expressive than 
C2 with respect to C, then Co is less expressive than £ 2 with respect to C. 

(Hi) If Co is less expressive than C\ with respect to C, then Co is less expressive than 
C\ with respect to C' . 

Proof. The proof is an easy calculation, verifying the conditions based on the above 
definitions. 1 

However, a uniform change to all languages can change expressiveness relations. 

Theorem 3.19 Expressiveness relationships are not invariant under uniform exten- 
sions of the languages. 

Proof. For a simple example, consider A, A n , and A„, and recall that the two sub- 
languages are incomparable by Propositions 3.9 and 3.15. To prove the claim, we 
uniformly add a begin construct, (begin e e), that evaluates two expressions in 
sequence and then discards the first value. The formal specification requires an ex- 
tension of the set of evaluation contexts to 



Now, A n + {begin} can (macro-) express X v x.e as X n x. (begin x e), but begin does 
not add anything to the power of A„: after all (begin e\ e 2 ) is (macro-) expressible 
as (X v xy .y)eie 2 in A„. Thus, in the extended setting A n + {begin} is more expressive 
than A„ + {begin}. 

The claim is still valid if the new facility is already in the language universe. Take 
the same example and add X v xy.y } i.e., Abramsky's [1] convergence tester C for A n , 
to both sub-languages, which is equivalent to adding begin. 1 



E :: 



(begin E e) 



and an additional reduction clause: 
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The example in the preceding theorem formalizes Algol 60's definition of call-by- 
value as an abbreviation of a call-by-name procedure preceded by an additional block 
or statement [30:12]; i.e., it is not the pure call-by-name subset of Algol that can 
define call-by- value but an extension thereof that includes a "strict" facility. The 
theorem thus shows how dangerous it is to use such informal claims as call-by-name 
can or cannot express call-by- value etc. These claims only tend to be true in specific 
language universes for specific conservative language restrictions: they often have no 
validity in other contexts! 



Syntax 



c 
/ 

V 



Semantics 

Evaluation Contexts 
Reduction Steps 



0 | 1 | — 1 | 2 | — 2 | ... (numerals) 

Or | 1 + | 1~ | + | — (numeric functions) 

c | / (constants) 

(lambda (x . . .) e) (abstractions) 

v (values) 

x (variables) 

(e e . . .) (applications) 

eval(e) holds iff e — >* v for some v 

E ::= a \ (v...E e . . .) 



E((fv...)) E(S(f,v,...)) \f6(f,v,...) is defined 

^(((lambda (x 1 . . .x n ) e) v 1 . . .v n )) — ► E(e[x 1 /v 1 , . . . , x n /v n ]) 



Constant Interpretation 

S(l + ,n) = n + 1 <*)( + , n, to) = n + to <*>(0r, 0) = (lambda (x y) x) 

S(l~,n) = n—1 S( — ,n,m) = n — m S(0T,n) = (lambda (x y) y) 

for n 0 



Figure 4: Pure Scheme 



4 The Structure of Idealized Scheme 

Pure Scheme is a simple functional programming language. It has multi-ary, call-by- 
value procedures and algebraic constants. There are basic constants and functional 
constants. Following Plotkin, we assume the existence of a partial function (6) from 
functional constants and closed values to closed values that specifies the behavior of 



24 



constants in Pure Scheme and its extensions. Typically, the constants include in- 
tegers, characters, booleans, and some appropriate functions; Figure 4 contains the 
appropriate definition of 8. In order to gain a complete understanding of Idealized 
Scheme, Pure Scheme only contains integers and a minimal set of functions on inte- 
gers, elements that are expressible in a A-calculus-based language like A„. 

Figure 4 contains the complete specification of Pure Scheme, based on a reduction 
semantics in the style of the previous section. Scheme programs satisfy two context- 
sensitive definitions: They are closed expressions, and they do not contain lambda- 
abstractions with repeated parameter names. The predicate eval holds for a program 
if the program reduces to an answer, that is, values in the case of Pure Scheme. If eval 
does not hold for a program, the program is either in an infinite loop or the reduction 
process is stuck. 11 In Pure Scheme, an evaluation can become stuck because of the 
application of a constant symbol to a A-expression, the application of a numeral to 
a value, the application of a constant function to a value for which 8 is undefined, 
or the application of a lambda-abstraction to the wrong number of arguments. As 
before, the reduction rules for Pure Scheme constitute the basis for proof systems for 
the operational equivalence relation in the spirit of Figure 3. For brevity, however, 
we shall carry out most of the proofs in this section in an informal setting; it should 
be clear from the proofs, though, how to formalize the various steps. In the following 
subsections, = ps denotes the operational equivalence relation on Pure Scheme; other 
indexes correspond to extensions of Pure Scheme and should be self-explanatory. 

The main characteristic of Idealized Scheme [11, 12, 13] is the extension of the func- 
tional core language Pure Scheme with type predicates, local branching constructs, 
and imperative facilities: 

• branching expressions for the local manipulation of control, 

• predicate constants for determining the type of a value, 

• control operators for the non-local manipulation of control, and 

• assignment statements for the manipulation of state variables. 

The extensions reflect the belief that these constructs increase the expressive power of 
the language [40, 41]. In this section, we demonstrate how to formulate these beliefs 
in our formal macro-expressiveness framework. 

Subsection 1 simultaneously deals with local control and type predicates because 
the two sets of constructs are closely related. The second subsection is a study of 
two different control operators, one for stopping the execution of a program and 
another for handling the general flow of control. The third subsection shows how 
imperative assignments add expressive power to the core language. Finally, the last 

11 Although this is common practice in semantic considerations, a more realistic specification would 
have to consider the introduction of an error mechanism. However, an error mechanism actually 
introduces additional expressive power, which is the reason why we consider it separately. 
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subsection addresses the unrelated issue of how Pure Scheme relates to so-called 
"lazy" functional languages, or more precisely, to call-by- value languages with call- 
by-name data constructors. We thus hope to reconcile Proposition 3.15 on the non- 
expressibility of call-by-name abstractions in A„ and Pure Scheme with the wide- 
spread belief that "lazy" data constructions are available in higher-order, by-value 
languages. 

4.1 Local Control and Dynamic Types 

The programming language world knows two types of local branching statements: the 
boolean- value based if-construct for distinguishing two values from each other, and 
the Lisp-style if-construct for distinguishing one special value from all others. The 
semantics of the former relies on the presence of two distinct values: false and true, 
or 0 and 1. Assuming an extension of the set of evaluation contexts to 



the following two additional reduction rules characterize the behavior of truth- value 
based Bif: 



If the test value in a Bif-expression is neither 0 nor 1, the evaluation of the program 
is undefined, or equivalently, such a Bif-expression is operationally indistinguishable 
from a diverging expression. The extension is obviously conservative; we refer to it 



Clearly, Pure Scheme can express such a simple Bif. 
Proposition 4.1 Pure Scheme can macro-express Bif. 

Proof Sketch. The proposition follows from Corollary 3.13 and is basically due to 
Landin and Burge [23:115], who realized that vacuous lambda abstractions could be 
used to suspend computations. Consider the syntactic abstraction: 

(Bif a a t aj) = ((lambda (t thn els) 

(((Or(l" t)) thn ((Or*) els (lambda () 0))))) 
a (lambda () ct t ) (lambda () aj)) 

It is easy to show that this abstraction is operationally equivalent to Bif. If the 
replacement for a is neither 0 nor 1, then both expressions diverge. Otherwise, both 
expressions select one of the replacements for a t or ctj and eliminates the other. 
The right-hand side accomplishes this by suspending the two expressions in 0-ary 
procedures and invoking one of them after the selection has been made, i 



E :: 



(Bif E e e), 





as PS(Bif). 
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Remark 8 (Weak Expressibility). An extension of Pure Scheme with Bif and 
two distinct new values, true and false, would not be macro-eliminable. Otherwise <p 
would have to map true to a term t in Pure Scheme, which implies that the programs 



(Bif true 1 2), 



and 



(Bif t 1 2), 



map to the same image, namely 



A(t,Lp(e 1 ),Lp(e 2 )) 



for some fixed syntactic abstraction A. But it is then impossible that the translation 
preserves program behavior because the first program terminates and the second one 
diverges. 

Clearly, Bif is still weakly expressible since the translation will only force more 
programs to terminate. In a typed version of Pure Scheme, the problem would disap- 
pear. The type system would not admit a program with an ill-typed Bif-expression. 
Put differently, since a typed version of Pure Scheme admits fewer programs, expres- 
siveness propositions are stronger, i 

The Lisp-style if assumes that there is one distinct value for false, in Lisp usually 
called nil, and all other values represent true. With 0 again serving as false, the 
reduction rules differ accordingly from (Bif.true) and (Bif.false): 



Proposition 4.2 Pure Scheme cannot macro-express Lif. 

Proof Sketch. For readability, we carry out the proof in P5(Bif). Since Bif is 
macro-expressible by Proposition 4.1, operational equivalences of terms hold in Pure 
Scheme after expanding the Bif-expressions. 

The proposition is a consequence of Theorem 3.14. The interesting operational 
equivalence is based on the following context: 



In Pure Scheme, the evaluation of (an instance of) this context cannot reach (the 
replacement of) a. First, if p is not bound to a procedure, the evaluation process 
diverges at the first invocation of p on (lambda () 0). Thus assume p is replaced 
by a procedure. The rest of the proof proceeds by a case analysis on the following 
property of procedures: a procedure of one argument may (1) ignore its argument and 
return a constant result, or (2) apply a constant function symbol to its argument, or 




e t for v ^ 0 






C(a) = (Bif (p (lambda () 0)) (Bif (p 0) 1 a) 0). 
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(3) use its argument in the procedure position of an application. For the evaluation 
of C(a) to reach a, the procedure must return two different results: 1 on (lambda 
() 0), and 0 on 0. Let us then consider the other two alternatives. On one hand, if 
p applies a functional constant in Pure Scheme to its argument, then the application 
(p (lambda () 0)) diverges. On the other, when p uses its argument as a procedure, 
the evaluation again diverges in the first test position. In short, either the evaluation 
of C diverges at the first test position, or the procedure p produces a result that is 
independent of its argument. Both cases imply that an evaluation cannot reduce a 
redex in the replacement of a. 

It follows from the above that it is inconsequential what a represents. Therefore, 
C(l) = ps C(O). In the larger setting of P5(Lif), however, the preceding analysis does 
not hold. A context over P5(Lif), could bind the variable p to the procedure 

(lambda (x) (Lif x 0 1)), 

which can distinguish the arguments 0 and (lambda () 0) in the correct manner: 

c(i)¥ Ltf c(n). . 

As an alternative to the addition of Lif, dynamically typed languages generally 
include type predicates. For extending Pure Scheme with a predicate symbol like 
int?, it suffices to extend the interpretation function 6 with the clauses 

<5(intr, c) = (lambda (x y) x) 
<5(intr, (lambda (. . .) e)) = (lambda (x y) y) 

Again, the extension, P^int?), is clearly conservative. 

With int?, programs in the extended language can now effectively test the type 
of a value, and indeed, int? can express Lif. It follows that int? is not expressible in 
Pure Scheme. 

Proposition 4.3 (i) Pure Scheme cannot express int?. 
(ii) P^int?) can express Lif. 

Proof Sketch. This proof illustrates the use of reduction proofs in the framework of 
expressiveness. First, P5(int?) can macro-express Lif: 

(Lif a a t aj) = (Bif (cand (intTa) (OT (1~ a))) a t aj), 

where 

(cand «i a 2 ) = (Bif «i a 2 0) 

and Bif is expressed as in Proposition 4.1 above. Second, since P5(int?) can express 
Lif, it is stronger than Pure Scheme by the preceding proposition, i 

The converse does not hold: A Lisp-style Lif can distinguish between 0 and all 
other values but not an arbitrary integer from the class of procedures. 
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Proposition 4.4 P.S'(Lif) cannot macro-express int?. 

Proof Sketch. The proof proceeds along the lines of the proof of Proposition 4.2. 
Instead of applying the procedure variable p to 0, the modified context invokes p 
on 1: 

C'(a) = (Bif (p (lambda () 0)) (Bif (p 1) 1 a) 0). 

The analysis uses the same reasoning as the above proposition with one exception: a 
procedure argument may now also appear in the test position of a Lif-expression. As 
above, for the evaluation of (an instance of) C to reach (the replacement of) a, the 
procedure may not invoke its argument, may not submit it to a constant function, 
but can test it with a Lif-expression. But this is irrelevant because both 1 and 
(lambda () 0) cause a Lif-expression to take the same branch. Hence, C"(l) =Lif 
C'(fi), yet, with int? in the language, this is no longer the case: C"(l) 9=mf? C'(Q). i 
Putting it all together, we see that Pure Scheme can handle some but not all types 
of local branching decisions. A simple, boolean- valued if construct is expressible. The 
more typical Lisp-style if adds the expressive power to distinguish one integer value 
from all other values, whereas the domain predicate intrpermits a distinction between 
each integer value and the class of all other values. 



Additional Syntax 

e ::= . . . | (call/cc e) (continuation captures) 

| (abort e) (program stops) 

Additional Reduction Steps 

£(call/cc e) — > E(e (lambda (x) (abort E(x)))) 
_E(abort e) — ► e 

Figure 5: Pure Scheme with control 



4.2 Non-Local Control 

A more interesting expressiveness constellation arises in the context of non-local con- 
trol abstractions. Idealized Scheme has the operations abort and call/cc. The former 
facility abandons the current evaluation context, realizing a simplistic form of error 
handling. The latter applies its subexpression to an abstraction of the current con- 
trol state, permitting almost arbitrary manipulations of the flow of control. Its name 
stands for "call with current continuation" because the Scheme-terminology refers to 
an abstraction of the control state as a "continuation" in analogy to denotational 
semantics. Figure 5 specifies the syntax and a simple reduction semantics of Pure 
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Scheme with both control operators. We refer to the entire extension as PS( control); 
=c+ a denotes its operational equivalence. Two interesting conservative restrictions of 
PS( control) are PS( abort) = Pure Scheme + abort and PS(ca\\/cc) = Pure Scheme 
+ call/cc with = a and = c as their respective operational equivalences. 

With the semantics of Figure 5, it is trivial to verify that the extensions are 
conservative over Pure Scheme. The semantics forms the basis of a simple equational 
calculus for abort and call/cc, and permits simple, algebra-like reasoning about 
programs with control operations [12, 13]. All three languages are more expressive 
than Pure Scheme. 

Proposition 4.5 Pure Scheme cannot macro-express non-local control constructs: 
Pure Scheme cannot macro-express abort or call/cc relative to PS( abort), PS(ca\\/cc) 
and P5( control) . 

Proof Sketch. The proof relies on Theorem 3.14, i.e., the addition of abort and 
call/cc invalidate operational equivalences over Pure Scheme. A typical example 12 
is the operational equivalence 

(lambda (/) ((/ 0) 0)) ^ ps (lambda (/) 0). 

As argued in the proof of Proposition 3 . 1 5 ( z ) , these two procedures are equivalent in 
a functional setting: both diverge when applied to a value. It is easy to check that 
this argument still holds in Pure Scheme. 

With abort and call/cc, however, there are contexts that invalidate this equiv- 
alence. Two examples are (a (lambda (x) (abort x))) and (call/cc a). Whereas 
the composition of the first expression with these contexts evaluates to 0, the second 
expression diverges in the same contexts: 

(lambda (/) ((/ 0) 0)) f x (lambda (/) 0). 

for x ranging over a, c, and c+a. i 

The next natural question is whether the two control operations are related or 
whether they provide distinct facilities. The following proposition shows that in 
Idealized Scheme, the two are actually independent enhancements of the expressive 
power of the core language. 13 

12 This example is a folk theorem example in the theoretical "continuation" community, but it was 
also used by Meyer and Riecke to argue the "unreasonableness" of continuations [26]. 

13 The non-expressibility of abort appears to be an artifact of our modeling of Scheme. A more 
realistic model of Scheme systems (as opposed to the Scheme semantics [35]) would have to include 
the interactive loop, which provides a delimiter for control actions [9]. By including an appropri- 
ate version of this delimiter in P5( control), abort becomes macro-expressible as a combination of 
call/cc and the control delimiter [39]. Put differently, interactive programming systems actually 
add expressive power to the programming language. Peter Lee [personal communication] pointed out 
another example of this phenomenon: The addition of a read-eval-print loop also introduces true, 
non-eliminable polymorphism into a language like A* + let by providing top-level let declarations 
with an open-ended body expression. The fact that such interactive programming environments add 
power to their underlying languages suggests that they should be specified as a part of the language 
standards! 
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Proposition 4.6 The control constructs abort and call/cc cannot express each 
other: 

(i) PS( abort) cannot macro-express call/cc with respect to PS( control) [39]. 
(ii) PS(ca\\/cc) cannot (macro-) express abort with respect to PS( control) . 

Proof Sketch. (i) The proof of the first claim shows that call/cc destroys operational 
equivalences in PS( abort). A typical example is C(l) = a C(0) where 

C(a) = (Bif (/ (lambda (k) ((k 1) 0))) 

(Bif (/ (lambda (k) ((k a) 0))) 0 1) 
0) 

These two terms could only differ if the procedure / invokes its argument, and if this 
invocation could return a result. In PS( abort), this is impossible because expres- 
sions can either produce a value, diverge, or abort. Therefore, the body of /'s first 
argument, (lambda (k) ((k 1) 0)), either aborts or diverges, but certainly cannot 
return a value. After adding call/cc, however, a context that binds / to 

(lambda (x) (call/cc x)) 

can distinguish the term C(l) from C(0): C(l) f c+a C(tt). 

(ii) The second claim is a consequence of Corollary 3.10, i.e., there is a program 
with an abort expression for which it is impossible to find an operationally equivalent 
call/cc expresion. The program is ((lambda (d) 0) (abort 1)); it is the composition 
of the context ((lambda (d) 0) a) over P5'(call/cc)and an abort expression. 

The absence of an operationally equivalent expression for (abort 1) from PS(ca\\/cc) 
follows from the property that expressions in the restricted language cannot eliminate 
their evaluation context. More technically, if E(e) is a program over PS( control) such 
that all occurrences of abort expressions have the form (abort E(e')) for some e', 
then either E(e) — >* E(v) or e = c + a ^- The proof of this auxiliary claim is a routine 
induction on n in the following statement: 

If E(e) — > n E(e') then either (1) e' is a value, or (2) e' contains a stuck 

redex, or (3) there is an e" such that E(e') > E(e") and E(e") satisfies 

the above condition. 

Since an expression over PS(ca\\/cc) does not contain any abort expression, it vac- 
uously satisfies the antecedent of the auxiliary claim. Hence, it either diverges or it 
returns a value and cannot be interchanged with an abort expression without effect 
on the behavior of a PS( control) program. 

From the existence of a program that contains (abort 1) and the non-existence 
of an operationally equivalent expression, it follows that PS(ca\\/cc) cannot express 
abort in PS( control), i 
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The preceding proposition not only establishes the formal expressiveness relation- 
ship among the control operators of Idealized Scheme , but it also provides a concrete 
example for the second claim in Theorem 3.14. 

Theorem 3.14 (restated) Let C\ = Co + {Fi, . . . , F n , . . .} be a conservative re- 
striction of Co. Let =o and =i be the operational equivalence relations of Co and C\, 
respectively. 

(ii) The converse of (i) does not hold. That is, there are cases where Co cannot 
express some facilities Fi, . . . , F n , . . ., even though the operational equivalence 
relation of C\ restricted to Co is identical to the operational equivalence relation 
of C 0 , i.e., = 0 = (=i \C 0 ). 

Proof. By the preceding proposition we know that PS(ca\\/cc) cannot express abort. 
To finish the proof, we only need to prove that the operational equivalence relation 
of PS(ca\\/cc) is a subset of the operational equivalence relation of PS( control): 
= c C (= c +a \ PS (c&W/cc)); the other direction is obvious. 

Assume that e 9= c+a e ' ' ■ We prove that e ^ c e 1 . Suppose there is a context C 
that can distinguish e and e 1 in PS( control). If the context is also a context over 
P^call/cc), the result is immediate. Otherwise, C contains a number of abort 
expressions, and there exists a context D(a, «i, . . . , ct n ) such that 

C(a) = D(a, (abort ei), . . . , (abort e n )). 

Now let a be a variable that does not occur in C, and let the context C"(e) be defined 
as follows: 

C'(a) = ((call/cc (lambda (a) 
(lambda () 

D(a, (a (lambda () ei)), . . . , (a (lambda () e n ))))))) 

Next, we show that eval(C(e)) holds if and only if eval(C'(e)) holds. First, the 
program C evaluates to an intermediate program with a few administrative steps: 

C'{e) — > + D(e, ((lambda (x) (abort (x))) (lambda () ei )), 
((lambda (x) (abort (x))) (lambda () e n ))) 

Second, by a generalized version of the call-by- value fi axiom, 

((lambda (x) (abort (x))) (lambda () e 8 -)) = c + a (abort e 8 -), 

and therefore 
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D(e, ((lambda (x) (abort (x))) (lambda () ei)), . . . , 
((lambda (x) (abort (x))) (lambda () e n ))) 

terminates if and only if 

D(e, (abort ei), (abort e n )) = C(e) 

terminates. The same analysis holds for the program C(e'), and we have thus shown 
that the context C'(ct) distinguishes e and e': e^ c e'. 14 i 

In summary, we have shown that PS( control) extends both PS( abort) and PS(ca\\/cc) 
with respect to expressive power, and the latter two individually extend Pure Scheme 
itself. An interesting point is that the extension of PS( abort) to PS( control) is qual- 
itatively different from the extension of PS(ca\\/cc) to PS( control). We expect this 
point to be a topic of further investigations. 

4.3 Assignments 

The final addition to Pure Scheme is the set!-construct, Scheme's form of assignment 
statement. Like in a traditional Algol-like programming language, the set!-expression 
destructively alters a binding of an identifier to a value. A simple reduction seman- 
tics for P.S'(state), Pure Scheme with set! and letrec (for recursive declarations of 
variables with initial values), is given in Figure 6. Clearly, P.S'(state) is a conservative 
extension of Pure Scheme; the new semantics is the basis for an equational calculus 
for reasoning about operational equivalences in P.S'(state) [11, 12]. 

Proposition 4.7 Pure Scheme cannot express set! and letrec. 

Proof Sketch. Consider the expression ((lambda (d) (/ 0)) (/ 0)), which contains 
the same subexpression, (/ 0), twice. In a functional language like Pure Scheme, the 
two subexpressions return the same value, if any, and, given that the value of the first 
subexpression is discarded, the expression is operationally equivalent to (/ 0): 

((lambda (d) (f 0)) (/ 0)) * ps (f 0). 

The verification of this equivalence in the proof system of Figure 3 is straightforward. 
In the extended language, this is no longer true. Consider the context 

C(a) = (letrec (/ (lambda (x) (set! / (lambda (x) 0)))) a), 

14 The transformation of C(e) into C(e') is not a homomorphic translation because it changes the 
top-level structure of the program. Since such a translation could encode a program as an integer 
and an interpreter as a function on the integers, a restricted language with all computable functions 
could express any feature if we allowed such global changes to programs. 



33 



Additional Syntax 



e 



(set! x e) 

(letrec ([x v ] . . .) e) 



(assignments) 

( recursive definitions) 



Extended Semantics 



eval(e) holds iff e 



* v or (letrec () e) — >* (letrec (. . .) v) for some v 



Additional Evaluation Contexts 



E::= ... | (set! x E) 



Additional Reduction Steps 



(letrec (...) E((Jv . . .))) 




(letrec (. . .) _E((lambda (x\ . . .) e) v\ . . .)) — ► (letrec (. . . [xi vi\ . . .) E(e[xj/vj, . . .])) 



(letrec (. . .[x v] . . .) E(x j) — > (letrec (. . . [x v] . . .) E(v j) 
(letrec (. . . [x u] . . .) i?((set! x v)) — ► (letrec (. . . [x v] . . .) E(v)) 
(letrec ([x v ] . . .) _E((letrec ([y u] . . .) e))) — ► (letrec ([x v ] . . . [y u] . . .) E(e)) 



which declares a procedure /. Upon the first application, the procedure modifies 
its declaration so that a second invocation leads to divergence. Consequently, an 
expression with a single use of the function converges, but an expression with two 
uses diverges: 



Not surprisingly, assignments increase the expressive power of Idealized Scheme. 
Without proof, we add that Scheme's form of assignment is equivalent to cells with 
a destructive update operation but without domain predicate. 

4.4 Non-evaluating Constructors 

Functional languages often use the call-by-name parameter-passing protocol instead 
of Pure Schemers call-by- value technique. Alternatively, such languages offer data 
constructors, say cons, that do not evaluate their arguments [15]. It is a widely held 



if the Xi, . . . are assignable 

and the x j , . . . are not assignable 



Figure 6: Pure Scheme with state 



((lambda (d) (f 0)) (/ 0)) f set[ (f 0). 
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belief that such provisions are superfluous in the presence of higher-order procedural 
abstractions. 



Evaluation Contexts 

E ::= a \ (u . . . E e . . .) where u = v \ {cons$} 

Additional Reduction Steps 

£((cons$ ei e 2 ) 1) — ► E( ei ) 
£((cons$ ei e 2 ) 2) E(e 2 ) 

Figure 7: Pwre Scheme(cons%) 



As shown in the previous section, call-by- value languages cannot express call- 
by-name abstractions. This result also holds in the extended framework of Pure 
Scheme. However, the introduction of non-evaluating data constructors is a bit more 
subtle. To study this issue more thoroughly, we consider two different conservative 
extensions of Pure Scheme, each of which incorporates a different form of a call- 
by-name constructor. The first extension, P^lazy), provides the constructor as a 
first-class function: 

v ::= ... | cons$ (call-by-name cons) 

| (cons$ e e) ( "lazy" values) 

For simplicity, "lazy" values are functions, and 1 and 2 serve as selector arguments. 
Figure 7 contains the corresponding extension of the reduction relation. Though 
not equivalent to full call-by-name abstractions, this addition of a single call-by-name 
primitive still introduces new semantic capabilities. A proof of this statement is easily 
derivable from Proposition 3.15. 

The second extension, PS( delayed), is a restriction of the first. The non-evaluating 
constructor is no longer a first-class function but is only available in first-order form: 

v ::= ... | (cons$ e e) ( "lazy" values) 

The reduction relation remains the same (Figure 7). It is this restricted extension 
that is expressible in Pure Scheme. 

Proposition 4.8 Pure Scheme can macro-express cons$ relative to PS( delayed) . 
Proof. The desired syntactic abstraction is 
(cons$ «i a 2 ) = (lambda (s) 

(Bif (or (r s)) ai (Bif (or (r (r s))) a 2 o))) 
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It is easy to check that the corresponding translation satisfies the reduction clauses 
of the original functions. The result follows from Corollary 3.13. i 

Remark 9 (Weak Expressibility). If the extended language contained selector 
functions for lazy values, the new values would only be weakly expressible for the 
same reason as Bif, true, and false are only weakly expressible (see Remark 8). i 

5 The Conciseness Conjecture 

If a programming language can represent all computable functions (on the integers), 
it contains a functionally equivalent counterpart to each program in a more expressive 
language. This raises the question as to what advantages there are to programming 
in the more expressive language when equivalent programs in the simpler language 
already exist. By the definition of an expressible construct, programs in a less expres- 
sive language generally have a globally different structure from functionally equivalent 
programs in a more expressive language. But, is this really all we can say about pro- 
gramming in more expressive languagesT 

By studying a number of examples, we have come to the conclusion that programs 
in less expressive languages exhibit repeated occurrences of programming patterns, 
and that this pattern-oriented style is detrimental to the programming process. To 
illustrate our point, we begin by presenting two examples. The first example compares 
two equivalent programs in variants of full Scheme and Scheme without assignment. 15 
Consider the following program fragment: 



... ( TransManager tl) . . . ) 

The program first binds the variable TransManager to a procedure that handles 
transactions and simultaneously counts how many transactions it performs. The 
procedure accomplishes the counting by allocating a local variable, TransCounter , in 
its private scope with initial value 0. For every subsequent proper transaction, the 

15 This comparison is part of the folklore of the expressiveness discussion [24:165]; the particular 
example is adapted from our previous paper on the equational semantics of assignments [11]. 



(let (. . . 



[TransManager (let (TransCounter 0) 

(lambda (TransType) 

(if (counter? TransType) 
TransCounter 
(begin 
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procedure then uses an assignment to increase TransCounter by 1. There is a special 
transaction of appropriate type that can check the number of past transactions. 

A program in Pure Scheme — or any other functional language without assignments — 
must realize the counting of transactions in a different way. For example, the above 
program fragment would have to be rewritten into something like the following code: 



This functional version of the program declares a variable for transaction counting in 
the same scope as the transaction manager procedure, which now takes the current 
value of the counter as an additional argument. Upon completion of the transaction, 
TransManager returns a pair whose first component is the increased counter value 
and whose second component is the proper result of the transaction. All calls to 
TransManager pass the current value of TransCounter as an extra argument. Finally, 
at every call site there is also some additional code to disassemble the result in the 
desired way. 

The functional version offers many opportunities for code simplifications. Specif- 
ically, every call site for the transaction procedure could immediately update the 
counter if the transaction is a proper transaction, and could return the value of the 
counter if the transaction causes a check on the number of previous calls: 



But, even after simplifying the functional version as much as possible, it always 
contains a large number of repeated occurrences of addl expressions, one per call site 
for TransManager , distributed over the whole program. 



(let (. . . 




(let (. . . 



[TransManager (lambda (TransType) BODY)] 
[TransCounter 0] 
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The second example concerns the use of control operators. Imagine a large func- 
tional program consisting of several modules. The interfaces of these modules have 
fully formal specifications in the form of (variants of) parameter descriptions. Now 
suppose that because of some extension of the program's requirements, one of the 
modules needs the capability to stop the execution of the (revised) program. In a 
functional setting, this task is accomplished by converting the relevant parts of the 
program into (simplified) continuation-passing style. Specifically, each function that 
(transitively) uses the critical module passes a functional abstraction of the rest of 
the computation to the critical module, and its call sites are in such a position that 
upon return, no further work needs to be done. It is thus up to the critical mod- 
ule to stop or to continue the execution of the rest of the program. If the former 
is necessary, the module discards the additional argument; otherwise, it invokes the 
argument on some intermediate result. This programming style, however, requires 
fundamental changes to the original, non- abortive program. First, the interface to 
the critical module must now indicate the possibility that the module could abort the 
program execution. Second, and more importantly, the code for every call site of a 
function with connections to the critical module must now satisfy special conditions. 
Again, as in the above example, there are alternatives, but for each of them, the lack 
of a non-expressible facility, this time the abort operation, causes the occurrence of 
programming patterns throughout the entire program. 

Based on these examples and others with a similar flavor, we have come to believe 
that the major negative consequence of a lack of expressiveness is the abundance 
of programming patterns to make up for the missing, non-expressible constructs. 
Clearly, a more specific conjecture about this issue must address the question of 
which programs actually benefit from the additional expressive power of larger lan- 
guages since not all of them do. A relatively naive answer would be that improved 
programs use non-expressible constructs in a sensible, observable manner. An exam- 
ple of a Scheme program that does not use assignments sensibly is a function whose 
only assignment statement occurs at procedure entry and affects the parameter. A 
more formal approach to the notion of "observable manner" could be the idea that 
a program with a sensible use of an additional feature must be transformable into a 
context that can witness operational distinctions between phrases in the restricted 
language. Despite the lack of a good definition for "sensible uses of constructs" or even 
for "programming patterns," we still venture to formulate the following conjecture 
about the use of expressive programming languages. 

Conciseness Conjecture. Programs in more expressive programming languages that 
use the additional facilities in a sensible manner contain fewer programming patterns 
than equivalent programs in less expressive languages. 

The most disturbing consequence of programming patterns is that they are an 
obstacle to an understanding of programs for both human readers and program- 
processing programs. In the above TransManager example, only a global program 
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analysis can verify that the addl expressions really count the number of transac- 
tions. Even worse there are two distinct explanations for a continuation-passing style 
subprogram in a call-by-name functional setting: it may either implement some so- 
phisticated control structure, or it may implement a call-by- value protocol [34]. Only 
a thorough analysis of the details of the continuation-passing program fragment can 
reveal the true purpose behind the occurrence of the programming patterns. Thus, 
the main benefit of the use of expressive languages seems to be the ability to abstract 
from programming patterns with simple statements and to state the purpose of a 
program in the concisest possible manner. 

6 Related Work 

The earliest attempt at defining and comparing the expressive power of programming 
languages is the work on comparative schematology by Chandra, Hewitt, Manna, 
Paterson, and others in the early and mid seventies [6, 32]. Schematology studies 
programming languages with a simple set of control constructs, e.g., while-loop pro- 
grams or recursion equations, and with uninterpreted constant and function symbols. 
As in predicate logic without arithmetic, it is possible to decide certain questions 
about such uninterpreted program schemas. Moreover, the languages are not univer- 
sal, and it makes sense to compare the set of functions that are computable based 
on different sets of control constructs, or based on an interpretation of a subset of 
the function symbols as operations on data structures like stacks, arrays, queues. In 
the presence of full arithmetic, i.e., representations of integers with an addition and 
multiplication function, the approach can no longer compare the expressive power of 
programming languages since everything can be encoded and all functions become 
computable. 

A second approach is due to Fortune et al [14]. Their basic observation is that 
statically typed languages without facilities for constructing diverging programs can 
only encode a subset of the total computable functions. For example, whereas the 
simply typed A-calculus-language can define the elementary recursive functions, the 
second-order version of the calculus comprises the e 0 elementary recursive functions. 
Like schematology, this approach crucially relies on the fact that the languages under 
consideration are not universal. While these two approaches illuminate some of the 
issues about the expressiveness of data and type structures, their applicability to full- 
fledged programming languages is impossible because an equating of expressiveness 
with computational power is uninteresting from the programmer's perspective. 

Recently, Hoare [20] proposed classifying programming languages according to 
the equational and inequational laws that their programming constructs satisfy. He 
illustrates this idea with a collection of examples. The laws are based on denotational 
semantics, which are generally sound with respect to operational equivalences. Given 
our theorems that connect expressiveness with the validity of operational equivalences 
in programming languages, this approach seems to be a related attempt at formalizing 
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or comparing the expressiveness of languages. 

Williams [45] looks at a whole spectrum of formalization techniques for seman- 
tic conventions in formal systems and, in particular, programming languages. His 
work starts with ideas of applicative and definitional extensions of formal systems 
but also considers techniques that are more relevant in computational settings, e.g., 
compilation and interpretation. The goal of Williams's research is a comparison of 
formalization techniques and not a study of the expressiveness of programming lan- 
guages. Some of his results may be relevant for future extensions of our work. 

A secondary piece of related work is the study of the full abstraction property of 
mathematical models [25, 31, 33] and the representability of functions in A-calculi [3, 
4]. In many cases, the natural denotational model of a programming language contains 
too many elements so that operationally equivalent phrases have different mathemat- 
ical meanings. Since it is relatively easy to reverse-engineer a programming language 
from a model, the equality relation of models without the full abstraction property 
directly corresponds to the operational equivalence of a conservative extension. As a 
consequence, such models naturally lead to the discovery of non-expressible program- 
ming constructs. In the framework of A-calculus languages, such facilities are multiple 
argument functions that do not require the values of all arguments to determine their 
result [33, 1]. Still, the study of full abstraction does not provide true insight into the 
expressive power of languages. On one hand, the discovery of new facilities directly 
depends on the choice of a model. For example, whereas a direct model of A n requires 
the above-mentioned facility for exploiting deterministic parallelism, a continuation 
model leads to operations on continuations and to restrictions of such operations [39]. 
On the other hand, by Theorem 3.14 we also know that a change in the operational 
equivalence relation is only a sufficient but not a necessary condition for the non- 
expressibility of a programming construct. In short, research on full abstraction is a 
valuable contribution to, but not a replacement for, the study of expressiveness (see 
Proposition 3.15). 

7 Towards a Formal Programming Language De- 
sign Space 

In the preceding sections we developed several ideas on a formal framework for com- 
paring the expressive power of programming languages. Based on informal claims in 
the literature, we argued that 

• the key to programming language comparisons is a restriction on the set of 
admissible translations between programming languages. 

Specifically, we proposed that 

• the translations between languages should preserve as much of a program's 
structure as possible. 
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An application ol this principle to conservative language extensions produced a num- 
ber of criteria for deciding whether additional operators increase the expressive power 
or not. For a concrete example, we considered several language extensions of Pure 
Scheme, a simple functional programming language, and found that our formal ex- 
pressiveness results are close to the intuitive ideas in the literature. 

The most important criterion for comparing programming languages showed that 
an increase in expressive power may destroy semantic properties of the core language 
that programmers may have become accustomed to (Theorem 3.14). Among other 
things, this invalidation of operational laws through language extensions implies that 
there are now more distinctions to be considered for semantic analyses of expressions 
in the core language. On the other hand, the use of more expressive languages seems 
to facilitate the programming process by making programs more concise and abstract 
(Conciseness Conjecture). Put together, this result says that 

• an increase in expressive power is related to a decrease of the set of "natural" 
(mathematically appealing) operational equivalences. 

An interesting challenge is to find expressive extensions of languages whose additional 
facilities do not invalidate operational laws. 16 

The current framework is only a first step towards a formal programming language 
design space. On one hand, we must investigate our comparison relation for arbitrary 
languages in more depth before we can judge its general usefulness. On the other 
hand, our set of restrictions on language translations is clearly not the only interesting 
basis for comparing programming languages. There is an entire spectrum of feasible 
restrictions that yield alternative notions of expressiveness, and these alternatives 
deserve exploration, too. Finally, we have not yet tackled the problem of deriving 
properties from expressiveness claims but expect to do so in the future. In the long 
run, we hope that some theory of language expressiveness develops into a formal 
theory of the programming language design space, and that such a theory can help a 
programmer in selecting the right set of constructs for solving a problem. 

Acknowledgement. Dan Friedman directed my attention to the idea of expres- 
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for parallel evaluations [2, 19] and single-threaded destructive updates in functional languages [18]. 
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